{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "RK255E7YoEIt"
   },
   "source": [
    "# DeepLabCut Toolbox - Docker\n",
    "https://github.com/DeepLabCut/DeepLabCut\n",
    "\n",
    "Nath\\*, Mathis\\* et al. *Using DeepLabCut for markerless pose estimation during behavior across species*\n",
    "\n",
    "This notebook demonstrates the necessary steps to use DeepLabCut on your own project.\n",
    "This shows the most simple code to do so, but many of the functions have additional features, so please check out the overview & the protocol paper!\n",
    "\n",
    "This notebook illustrates how to use the Docker container to:\n",
    "- train a network\n",
    "- evaluate a network\n",
    "- analyze a novel video\n",
    "\n",
    "This assumes you already have a project folder with labeled data! \n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "txoddlM8hLKm"
   },
   "source": [
    "## Let's look at info about the Docker Environment:\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "4C5WRoS9g5Od"
   },
   "outputs": [],
   "source": [
    "!nvcc --version"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "HxVNyimFp-PJ"
   },
   "outputs": [],
   "source": [
    "import tensorflow as tf\n",
    "tf.__version__"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "Pm_PC1Q8lRrH"
   },
   "outputs": [],
   "source": [
    "#let's make sure we see a GPU:\n",
    "#tf.test.gpu_device_name()\n",
    "#or\n",
    "from tensorflow.python.client import device_lib\n",
    "device_lib.list_local_devices()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Start here for training DeepLabCut and analyzing new videos!"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "sXufoX6INe6w"
   },
   "outputs": [],
   "source": [
    "#GUIs don't work on in Docker (or the cloud), so label your data locally on your computer! \n",
    "#This notebook is for you to train and run video analysis!\n",
    "import os\n",
    "os.environ[\"DLClight\"]=\"True\""
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "3K9Ndy1beyfG",
    "scrolled": true
   },
   "outputs": [],
   "source": [
    "# now we are ready to train!\n",
    "import deeplabcut\n",
    "deeplabcut.__version__"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### change to your path:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "Z7ZlDr3wV4D1"
   },
   "outputs": [],
   "source": [
    "path_config_file = '/home/mackenzie/DEEPLABCUT/DeepLabCut2.0/examples/Reaching-Mackenzie-2018-08-30/config.yaml' #change to yours!"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Note, if you moved the project, or downloaded this and using the demo code, you will need to edit the project path in the config.yaml file! \n",
    "\n",
    "Head over to the project folder to open the yaml file in any text editor (such as gedit in Ubuntu)\n",
    "\n",
    "(description): project_path:  Full path of the project (edit if you need to move the project to a cluster/server/another computer or a different directory on your computer)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "xNi9s1dboEJN"
   },
   "source": [
    "## Create a training dataset\n",
    "This function generates the training data information for DeepCut (which requires a mat file) based on the pandas dataframes that hold label information. The user can set the fraction of the training set size (from all labeled image in the hd5 file) in the config.yaml file. While creating the dataset, the user can create multiple shuffles. \n",
    "\n",
    "After running this script the training dataset is created and saved in the project directory under the subdirectory **'training-datasets'**\n",
    "\n",
    "This function also creates new subdirectories under **dlc-models** and appends the project config.yaml file with the correct path to the training and testing pose configuration file. These files hold the parameters for training the network. Such an example file is provided with the toolbox and named as **pose_cfg.yaml**."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "deeplabcut.create_training_dataset(path_config_file,Shuffles=[1])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### now go edit the pose_cfg.yaml to make display_iters: low (i.e. 10), and save_iters: 500 (for demo's)\n",
    "\n",
    "Now it is the time to start training the network!"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "c4FczXGDoEJU"
   },
   "source": [
    "## Start training\n",
    "This function trains the network for a specific shuffle of the training dataset. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "_pOvDq_2oEJW",
    "scrolled": false
   },
   "outputs": [],
   "source": [
    "#reset in case you started a session before...\n",
    "#tf.reset_default_graph()\n",
    "\n",
    "deeplabcut.train_network(path_config_file, shuffle=1, saveiters=1000, displayiters=10)\n",
    "\n",
    "#this will run until you stop it (CTRL+C), or hit \"STOP\" icon, or when it hits the end (default, 1.3M iterations). \n",
    "#Whichever you chose, you will see what looks like an error message, but it's not an error - don't worry....\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Firstly, if the above cell ran, you can stop it with \"stop\" or cntrl-C; you will get a Keyboard Interrupt error (this is fine!)\n",
    "\n",
    "### A couple tips for possible troubleshooting (1): \n",
    "\n",
    "if you get **permission errors** when you run this step (above), first check if the weights downloaded. As some docker containers might not have privileges for this (it can be user specific). They should be under 'init_weights' (see path in the pose_cfg.yaml file). You can enter the DOCKER in the terminal:"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "see more here: https://github.com/MMathisLab/Docker4DeepLabCut2.0#using-the-docker-for-training-and-video-analysis"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "You can \"cd\" in the terminal to this location! i.e. copy and paste this in: **\"cd usr/local/lib/python3.6/dist-packages/deeplabcut/pose_estimation_tensorflow/models/pretrained/\n",
    "\"** \n",
    "\n",
    "And if you type \"ls\" to see the list of files, you should see the resnet:\n",
    "**resnet_v1_50.ckpt**\n",
    "\n",
    "If it is not there, run **\"sudo download.sh\"**\n",
    "then change the permissions: **\"sudo chown yourusername:yourusername resnet_v1_50.ckpt\"**\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Troubleshooting (2): \n",
    "if it appears the training does not start (i.e. \"Starting training...\" does not print immediately),\n",
    "then you have another session running on your GPU. Go check \"nvidia-smi\" and look at the process names. You can only have 1 per GPU!)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "xZygsb2DoEJc"
   },
   "source": [
    "## Start evaluating\n",
    "This function evaluates a trained model for a specific shuffle/shuffles at a particular state or all the states on the data set (images)\n",
    "and stores the results as .csv file in a subdirectory under **evaluation-results**"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "nv4zlbrnoEJg",
    "scrolled": false
   },
   "outputs": [],
   "source": [
    "deeplabcut.evaluate_network(path_config_file)\n",
    "\n",
    "# Here you want to see a low pixel error! Of course, it can only be as good as the labeler, so be sure your labels are good!"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "BaLBl3TQtrfB"
   },
   "source": [
    "## There is an optional refinement step\n",
    "- if your pixel errors are not low enough, please check out the protocol guide on how to refine your network!\n",
    "- You will need to adjust the labels **outside of DOCKER!** We recommend coming back to train and analyze videos... \n",
    "- Please see the repo and protocol instructions on how to refine your data!"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "OVFLSKKfoEJk"
   },
   "source": [
    "## Start Analyzing videos\n",
    "This function analyzes the new video. The user can choose the best model from the evaluation results and specify the correct snapshot index for the variable **snapshotindex** in the **config.yaml** file. Otherwise, by default the most recent snapshot is used to analyse the video.\n",
    "\n",
    "The results are stored in hd5 file in the same directory where the video resides. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "Y_LZiS_0oEJl",
    "scrolled": false
   },
   "outputs": [],
   "source": [
    "videofile_path = ['/home/mackenzie/DEEPLABCUT/DeepLabCut2.0/examples/Reaching-Mackenzie-2018-08-30/videos/MovieS2_Perturbation_noLaser_compressed.avi'] #Enter the list of videos to analyze.\n",
    "deeplabcut.analyze_videos(path_config_file,videofile_path)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "pCrUvQIvoEKD"
   },
   "source": [
    "## Create labeled video\n",
    "This function is for visualiztion purpose and can be used to create a video in .mp4 format with labels predicted by the network. This video is saved in the same directory where the original video resides. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "6aDF7Q7KoEKE",
    "scrolled": false
   },
   "outputs": [],
   "source": [
    "deeplabcut.create_labeled_video(path_config_file,videofile_path)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "8GTiuJESoEKH"
   },
   "source": [
    "## Plot the trajectories of the analyzed videos\n",
    "This function plots the trajectories of all the body parts across the entire video. Each body part is identified by a unique color."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "gX21zZbXoEKJ",
    "scrolled": false
   },
   "outputs": [],
   "source": [
    "%matplotlib notebook \n",
    "#for making interactive plots.\n",
    "#deeplabcut.plot_trajectories(path_config_file,videofile_path, plotting=True)\n",
    "\n",
    "deeplabcut.plot_trajectories(path_config_file,videofile_path,showfigures=True)"
   ]
  }
 ],
 "metadata": {
  "accelerator": "GPU",
  "colab": {
   "collapsed_sections": [],
   "name": "Colab_TrainNetwork_VideoAnalysis.ipynb",
   "private_outputs": true,
   "provenance": [],
   "toc_visible": true,
   "version": "0.3.2"
  },
  "kernelspec": {
   "display_name": "Python [default]",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.6.6"
  },
  "varInspector": {
   "cols": {
    "lenName": 16,
    "lenType": 16,
    "lenVar": 40
   },
   "kernels_config": {
    "python": {
     "delete_cmd_postfix": "",
     "delete_cmd_prefix": "del ",
     "library": "var_list.py",
     "varRefreshCmd": "print(var_dic_list())"
    },
    "r": {
     "delete_cmd_postfix": ") ",
     "delete_cmd_prefix": "rm(",
     "library": "var_list.r",
     "varRefreshCmd": "cat(var_dic_list()) "
    }
   },
   "types_to_exclude": [
    "module",
    "function",
    "builtin_function_or_method",
    "instance",
    "_Feature"
   ],
   "window_display": false
  }
 },
 "nbformat": 4,
 "nbformat_minor": 1
}
